The MALTASE Framework For Usage-Aware Software Evolution

Software developers need to know how their software is used to improve it and adjust it to user needs. Helpful usage information includes failure reproduction steps, user skill levels, and deviations of actual usage from expected usage. However, this information is rarely available to developers because of communication gaps between users and developers. Therefore, developers are often unaware of failures and problems users are facing, make potentially wrong assumptions about software usage, and cannot reflect the user perspective in their decisions. This dissertation investigates the acquisition of usage information by monitoring and analyzing user interactions. It describes the Maltase framework which monitors high-level user interactions and analyzes them to acquire usage information which is helpful for developers during software evolution tasks. In a problem case study, we found that developers need the following usage information during software evolution: use cases and user behavior, user goals and user needs, failure reproduction steps, and application domain concepts. We demonstrate the applicability of Maltase by implementing three ways of exploiting monitored, high-level user interactions: provision of failure reproduction steps, classification of users according to their skills, and detection of deviations between user behavior and use case steps. To investigate the impact of Maltase, we conducted a controlled experiment and an evaluation case study. The controlled experiment compared failure reproduction with interaction traces to failure reproduction with textual bug reports. It found that developers can reproduce failures based on interaction traces and that inexperienced developers are enabled to reproduce failures they cannot reproduce with textual bug reports alone. The evaluation case study learned skill classifiers from interaction traces of participants with differing skill levels. It found that skill classifiers are able to reliably discriminate between novice and expert users of a specific application (86–93% accuracy) but not between domain skill levels (64–71% accuracy). In a simulation and a user study, we found that Maltase introduces an overhead of 5 % execution time and that users tolerate this overhead. Overall, our evaluation showed that Maltase narrows communication gaps between users and developers by automatically acquiring usage information.